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Abstract 

In this paper, we accomplish two objectives: First, we provide a new mathematical characterization 
of the value function for impulse control problems with implementation delay and present a direct 
solution method that differs from its counterparts that use quasi- variational inequahties. Our method 
is direct, in the sense that we do not have to guess the form of the solution and we do not have to 
prove that the conjectured solution satisfies conditions of a verification lemma. Second, by employ- 
ing this direct solution method, we solve two examples that involve decision delays: an exchange 
rate intervention problem and a problem of labor force optimization. 
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1 Introduction 

Implementation delays occur naturally in decision-making problems. Many corporations face regulatory 
delays, which need to be taken into account when the corporations make decisions under uncertainty. 
A decision made will be carried out only after certain amount of time elapses, for example, due to 
regulatory reasons. The decision involves optimally exercising a real option or optimally manipulating 
(with some associated cost) a state variable, which is the source of uncertainty. Several problems that fit 
into this framework can be found in the literature: The work of Bar-Ilan and Strange ||6l constitutes the 
first study considering how delays affect rational investment behavior. Keppo and Peura lUTl consider 
the decision making problem a bank has to solve when it is faced with a minimum capital requirement, 
a random income, and delayed (and costly) recapitalization. The bank's problem is to determine when 
to raise capital from its shareholders and the amount to be raised, given that this transaction requires a 
heavy preparatory work, which causes delay. Bar-Ilan and Strange f7| consider (irreversible) sequential 



*E. Bayraktar was supported in part by the National Science Foundation, under grant DMS-0604491. 
^E. Bayraktar and M. Egami are in the Department of Mathematics, University of Michigan, Ann Arbor, MI 48109, USA, 
email: {erhan, egami} @umich.edu. 



1 



(2 stage) investment decision problems given two sources of delay: one due to market analysis in the 
first stage and the other due to construction of a production facility in the second stage. In each stage the 
firm's problem is to decide whether to continue entering into the market (of that product) or to abandon 
it. See also Subramanian and Jarrow 1241 who consider the problem of a trader (who is not a price taker) 
who wants to liquidate her position and encounters execution delays in an illiquid market. Alvarez and 
Keppo IS study the impact of delivery lags on irreversible investment demand under revenue uncertainty. 
0ksendal et. al. |[20l . |[T5l consider the classical stochastic control of stochastic delays systems. 

The problem of finding an optimal decision (in the presence of delays) can be characterized as a 
stochastic impulse control problem or an optimal stopping problem. In the papers cited above the im- 
pulse control problem or the optimal stopping problem were solved by using a system of quasi- variational 
inequalities. (See e.g. Bensoussan and Lions HI and 0ksendal and Sulem |[2T]| for the relationship be- 
tween control problems and quasi-variational inequalities.) In a different approach, 0ksendal and Sulem 
II22I solve a version of delay problems, in which the controller decides on the magnitude of control at 
the time of decision-making before any delay (the decision is implemented after some delay). They con- 
vert the optimal impulse control problem with delayed reaction into a no-delay optimal stopping/impulse 
control problem. Note that choosing the control in this way introduces strong path dependence of the 
controlled process. 

Here, we solve the impulse control problems with delays directly and the magnitude of the impulses 
are chosen at the time of action, not at the time of decision-making, by providing a new characterization 
of the value function. The controlled process is a non-Markov process in this case, too, since depending 
on when a point in the state space is reached, it has different roles. But the controlled process in this 
case regenerates after a decision is implemented, and the value of the state process during the delay 
time depends on the past only through the value of the state process at the time of decision-making. We 
will only consider the threshold and band policies in this paper, since we expect that the non-Markovian 
structure will make finding the optimal solution much more difficult if we allow more general strategies. 
For example, because of the lack of Strong Markov property, we were unable to prove the concavity 
properties of the value function when the admissible strategies were a superset of band or threshold 
strategies. 

Our results rely on the works of Dynkin lfT3l . lfT4l (see e.g. Theorem 16.4) and Dayanik and Karatzas 
|[T2l . who give a general characterization of optimal stopping times of one dimensional diffusions, and 
on the work of Dayanik and Egami lITTI . who characterize the value function of stochastic impulse 
control problems. Our method is direct, in the sense that we do not have to guess the form of the 
solution and we do not have to prove that the conjectured solution satisfies conditions of a verification 
lemma as all the methods in the above literature do. Other works similar in vein to ours that provide 
different characterizations of the value function of impulse/singular control problems for one dimensional 
diffusions rather than solving variational inequalities are Alvarez ID, El; Alvarez and Virtanen f?]; and 
Weerasighe Ii25il . 

We give a geometric characterization of the value function, specifically, we find very general con- 
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ditions on the reward function and the coefficients of the underlying diffusion under which the value 
function can be linearized (in the continuation region) after a suitable transformation. Then the prob- 
lem of determining the value function is equivalent to determining the slope (if admissible strategies are 
threshold strategies), the slope and the intercept (if admissible strategies are band strategies) from first 
order conditions. To show the efficacy of our methodology we apply it to an optimization problem of 
a central bank that needs to carry out exchange rate intervention (this is the Krugman model of interest 
rates considered, among others, in Mundaca and 0ksendal ifTSl ) when there is delay in the implemen- 
tation of its decisions. Also, using our methodology we will find optimal hiring and firing decisions 
of a firm that faces stochastic demand and has to conform to regulatory delays. Other works that deal 
with labor optimization problems are Bentolila and Bertola ||9l, and Shepp and Shiryaev ll23l who model 
firing and hiring decisions as singular controls. It is also worth pointing out that an impulse control study 
when the underlying process is a superposition of a Brownian motion and a compound Poisson process 
(when the jumps are of phase type) is given by f5^ with management of foreign exchange reserves and 
labor optimization in mind. 

The rest of the paper is organized as follows: In Section 2, we give a characterization of general 
threshold strategies with implementation delays and provide an easily implemented algorithm to find the 
value function and the optimal control. To illustrate our methodology, we will solve a delayed version 
of an example from Mundaca and 0ksendal ifTSl (also see 0ksendal |[T9l ). A similar problem to the 
one we consider was solved in ||22]1 in which the controller decides on the magnitude of control at the 
time of decision-making before any delay. In Section 3, we work with a band policy. In this section 
we work on the specific example of optimal hiring and firing decisions rather than providing a general 
characterization for the value function. We again provide an easily implemented algorithm to find the 
optimal control. Finally, we conclude in Section 01 



2 Optimal Threshold Strategies 

Let {Q,J^,F) be. a. complete probability space with a standard Brownian motion W = {Wt;t > 0} and 
consider the diffusion process with state pace X = {c,d) CM and dynamics 

dX^ = n{X^)dt + a{X^)dWt (2.1) 

for some Borel functions /i : T — > M and o" : T ^ (0, oo). (We assume that the functions and a 
are sufficiently regular so that (12.11 ) makes sense.) Here we take c and d to be a natural boundaries. We 
use "0" as the superscript to indicate that X^ is the uncontrolled process. We denote the infinitesimal 
generator of X^ by A and consider the ODE {A — a)v{x) = 0. This equation has two fundamental 
solutions, V'(-) and (^(•). We set V'l") ^ be the increasing and </?(•) to be the decreasing solution. = 
0, 95(0+) = 00 and ip{d—) = 00, ip(d—) = because both c and d are natural boundaries. First, we 
define an increasing function 

F{x) ^ (2.2) 
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Next, following (T?!, p. 238, we define concavity of a function with respect F as follows: A real valued 
function u is called F -concave on (c, d) if, for every c < I < r < d and x £ [l,r], 

/ ^ ^ n^F{r)-F{x) , ,F{x)-F{l) 

Suppose that at any time t £ M+ and any state x G we can intervene and give the system an 
impulse ^ G M. Once the system gets intervened, the point moves from x to y £ M+ with associated 
reward and cost. An impulse control for the system is a double sequence, 

u = {Ti,T2,....Ti...-^i,C2,.:Ci--) (2.3) 

where < Ti < T2 < .... is an increasing sequence of F-stopping times such that Tj+i — Tj > A, and 
^1, .^2--- are J^(Ti+A)- measurable random variables representing impulses exercised at the corresponding 
intervention times Tj with £ Z for all i where Z C M is a given set of admissible impulse values. The 
controlled process until the first intervention time is described as follows: 

dXt = n{Xt)dt + a{Xt)dWt, < t < Ti + A 

(2.4) 

Xt^+A = r(^(Ti+A)-;6) 

with some mapping T : (c, d) x M ^ M. We consider the following performance measure associated 
with 1/ G V (= a collection of admissible strategies). 



rix) = 



POO 
Jo 



Ti<oo 



(2.5) 



The objective (we shall call it the delay problem) is to find the optimal strategy z^* (if it exists) and the 
value function: 

v{x) = sup r{x) = J"* (x). (2.6) 

Remark 2.1. The controlled process X is not a Markov process, since depending on whether a point is 
reached in the time interval [Tj, Tj + A) or not, that point has dijferent roles. (The controlled process 
might jump or not at a given point depending on how it reaches to that point.) However, 1) the process 
regenerates at times {Ti + Ajjgpj) <^nd 2) the value of the process at time T £ {Ti,Ti^/\), Xj-, depends 
on the information up to Ti, J^Xi, only through the value of the process at time Ti, Xt^. Instead of finding 
the optimal strategy for a non-Markov process, we will use the hints of Markovian features to find the 
optimal threshold strategy ( see Assumption 12. 1 D . 

The following is a standing assumption in Sections 2. 1 and 2.2. 
Assumption 2.1. We make the following assumptions in this section: 

(a) We will assume that the set of admissible strategies is limited to threshold strategies. These strate- 
gies are determined by specifying two numbers a G (c, d) and h G (c, d) as follows: At the time 
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the uncontrolled process hits level b, the controller decides to reduce the level of the process from 
^Ti- = b to a < b, through an intervention, and save the continuously incurred cost (which is high 
if the process is at a high level). But the implementation of this decision is subject to a delay of A 
units of time. Note that ^[Ti+A)~ might be less than a. In that case the impulse applied increases 
the value of the process. Otherwise, if the value of the process is greater than a at time (Tj + A) — 
then the intervention reduces the level of the process to a. 



(b) The running cost function / : (c, d) ^ M is a continuous functions that satisfies 



(c) For any point x S (c, d), we assume 



< oo. 



K{x,x) < 0. 

We make this assumption to account for the fixed cost of making an intervention. 



(2.7) 



(2.8) 



2.1 Characterization of the Value Function 



In this section, we will show that when we apply a suitable transformation to the value function cor- 
responding to a particular threshold strategy (that is identified by a pair (a, b)), the transformed value 
function is linear on (0, F{b)). This characterization will become important in determining the optimal 
threshold strategy in the next section. 



Let us define 



g{x) ^ 



■ POO 

Jo 



(2.9) 



The following identity, which can be derived using the Strong Markov Property of X^, will come handy 
in a couple of computations below: 



E" 



for any stopping time r under the assumption (12.7 



Jo J 



(2.10) 



Now, let us simplify J'^ by splitting the terms in (12.51 ). We can write the first terms (the term with the 
integral) as 

" /"OO 

E^ 



poo 

Jo 



E^ 



Ti+A 



5(x)-E-[e-(^i+^)5(^^,+A)]+]E^ 



r r /"°° 

L Uo 



(2.11) 
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while the second term can be developed as 



.Ti<oo 



i=2 



e-"(^^+^)i^(X(T,+A)-, ^t,+a) + e-"(^^+^)E- 



((T,+A)oe(Ti+A)) 



= E^ 



,-a(Ti+A) 



^e-"(^'+^)i^(X(^.+A)-,^T,+A) 



where we used T^+i + A = (Ti + A) + (T^ + A) o e{Ti + A) with the shift operator 0{-) in the second 
equality. Here, we relied on Remark IZT] Combining the two terms, we can write (12.51 ) as 

r{x) = [e-"(^i+^) {K{X(^t.+A)-,Xt^+a) - 5(^m+A)-) + J'^(Xt,+a)}] + g{x). 
We define 



u = J - g. 

By adding and subtracting ^(^{Ti+A)) to ^rid from the first term we obtain 

u{x) = [e-"(^i+^)i?(X(r,+A)-,^Ti+A) + ^x(Xti+a) 



in which 



K{x, y) = K{x, y) - g{x) + g{y). 
Since Ti— = T}, with rj, = inf{t > : Xj^ > b} and the post intervention point by 



^Ti+A — ^n+A - -'^(r^+A)- - 6 



a. 



(2.12) 



(2.13) 



(2.14) 



(2.15) 



From Remark |2?T] 



u{x) = 



{K{Xr,+A,a)+u{a)} 



E^ 



-a{n+A) 



{K{Xr,+A,a) + uia)} 



Tb 



= E^ [e-"^^E^-i> [e-°^ {i^(XA,a) + n(a)}]] . (2.16) 
Evaluating at x = 6, we obtain u{h) = E^[e~°^ {K{X/A,a) + u{a)]]. Therefore, (I2l3]) becomes 

n(x) = E^ [e-"^''u(X^J] . 

Hence we have finally 



u{x) 



no(x) = E^ [e-"^''n(6)] , x G (c, 6), 

E^ [e-"^(i?(XA,a) +^xo(a))] , x G 



(2.17) 
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where the second equaUty is obtained when we plug Ti = in (12.13b . 

Using appropriate boundary conditions one can solve {A — a)u = and obtain 

for X G [l,r] where r; = mi{t > 0;X° = 1} and = inf{i > 0;X^ = r} (see e.g. Dayanik and 
Karatzas HH). By defining 

W = {u/ip)oF-\ (2.19) 

equation (12.171 ) becomes 

H^(F(x)) = Ty(F(c)) : + : , xE(c,6], (2.20) 
We should note that F{c) = F{c+) = V'(c+)/(/?(c+) = and 

W{F{c)) = lc = lim sup Ei^l^ (2.21) 

xlc ^{X) 

for any a G (c, d). For more detailed mathematical meaning of this value Ic, we refer the reader to 
Dayanik and Karatzas 1121 . We have now established that W{F{x)) is a linear function in the trans- 
formed "continuation region". 



2.2 An Algorithm to Compute the Value Function 

Let us denote 

r{x; a) = E^[e-"^i?(XA, a)] (2.22) 

and transform this function by 

First stage: For a given pair (a, 6) G (c, x (c, d) we can determine (12.171 ) from the linear charac- 
terization (12.201 ). On (0, F{b)\ we will find W{y) = py + Ic (in which the slope is to be determined) 
from 

pF{b) +k = R{F{b),a) + e-'^^ipFia) + Q"^. (2.24) 

p can be determined as 

i^(F(b;a)) + /.(e-"Agg-i) 
/O = n — ^ (2-25) 

Sometimes we will refer to p as 6 — > /9(6), when it becomes necessary to emphasize the dependence on 
h. The function u can be written as 

J uq{x) = pipix) + /cV^(x) x<b 
u{x) = < (2.26) 
\r{x,a) + e " no(a) x > b. 
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Note that (A — a)u{x) = for x < b. Henceforth, to emphasize the dependence on the pair (a, b) we 
will write u"''^{-)for the function u{-). 

Second stage: Our purpose in this section is to determine 

u"(x) = sup (x), xG(c,d), (2.27) 

be(c,d) 

to determine the constant b* 

u"" (x) = u"'^* (x) , x G (c, d) , (2.28) 

if there exists one. 

Let us fix a and treat /? as a function of b parametrized by a. 

Lemma 2.1. Assume that the function R{-;a) defined in ([2.23i is differentiable and that there exists a 
constant b* S (c, d) satisfying l \2.28i . Then b* satisfies the equation 



pF'{b) = ^R{y-a) 



F\b)-e-HpF{a) + Q^^^^. (2.29) 
y=F{b) ^ibf 



in which p is given by d2.25D . 



Proof. From (12.261) it follows that the maximums of the functions b —>■ u""'^ and b p{b) are attained at 
the same point. Now taking the derivative of ( 12.241) and evaluating at pf, = we obtain (12.291 ). □ 

To find the optimal b (given a) we solve the non-linear and implicit equation ( 12.291 ). Under certain 
assumptions on the function {r /(p) o F^^, this equation has a unique solution as we show below. 

Remark 2.2. Ony > F{b), the function W is given by 

Wiy) = e-'^^ipFia) + + R(y; a). (2.30) 

The right derivative ofW at F(b) is given by 

(2.31) 



w'iFm ^ -.-(.n-) + + ^^^^^^ 

Therefore, l \2.29i implies that the left and the right derivative ofW (recall that W{y) = py + Icfor 
y < F{b)) at F{b) are equal (smooth fit). 

Let us define 

Uaix) ^ sup [e-""''E^-i> [e-"^ {^(XA,a) + Uaia)}]] . (2.32) 
fee(c,d) 

The next lemma shows that (12.321 ) is well-defined. Below we show that under certain assumptions on 
{r/(p) o F~^ this function is equal to n". 
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Lemma 2.2. Assume that 



sup E^[i^(XA,a)] > (2.33) 



for some a G (c, d). Let us introduce afamUy of value functions parameterized by ^ as 

yj(x) = supE^ \e-<^+^) {i^(X°+A, a) + 7}] = supE^ [e-""E^° [e""^ {^(^A, + 7}] 



re5 



r65 



(2.34) 

/lere 5 the set of all stopping times of the filtration natural filtration of X^. Then there exists a unique 
7* such that (a) = 7*. 

Proof Let us denote 

W2{F{x)) ^ (2.35) 

Consider the function 7 ^ V^7(a). Our aim is to show that there exists a fixed point to this function. 
Let us consider V^{a) first. Because (12.331) is satisfied we have that V^{a) > 0. As 7 increases, V'^{a) 
increases monotonically, by the right hand side of (12.341 ). Now, Lemma lSTTl implies that for 71 > 72 > 0, 

K7K^)-K7^(^)<7i-72 (2.36) 

for any x G M+. Note that W2{F{a)) > R{F{a),a) + ^^^^ for all 7. However, since V has less than 
linear growth in 7 as demonstrated by (12.361 ) we can see that there is a certain 7 large enough such that 
W2{F{a)) = R{F{a),a) + for 7 > 7'. This implies however 

^{a)W2' (F(a)) = ^{a)R{F{a) , a) + e'^'^Y 
^y/(a)=r(a,a) + e-"V<7' 

where the inequality is due to the assumption (12.81) . For this 7 , we have Va (a) < 7 . 

Since 7 — > K7 is continuous, which follows from the fact that this function is convex, and increasing, 
> and Va (a) < 7 imphes that 7 — > crosses the line 7^7. 

□ 

Lemma 2.3. Assume that 

r{x,a) is lower semi-continuous. (2.37) 
Let us define R^{-;a) = where 

r^{x, a) ^ E"[e-°^(K(XA, a) + 7)]. (2.38) 

r/ie?i ( 12.551 ) « smallest non-negative concave majorant of that passes through {F{c+),lc)- 

Proof. See for e.g. Dynkin lfT4l and Dayanik and Karatzas |[T2l . □ 
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Lemma 2.4. Assume that d2. jJD and ([2.37\l hold. Then Uaj'-p is F— concave, i.e., a— excessive^ 



Proof. This follows from Lemmas 12.21 and 1231 For the equivalence of a-excessivity and F— concavity 
see e.g. Theorem 12.4 in [14| and also [12]. This fact can be observed from (15.81 ). □ 



Lemma 2.5. Assume that d2. jJD and Ii2.37\) hold. Then 

u"'{x) < Ua{x), X G (c, d). 



(2.39) 



Proof. It follows from Lemma l2!4l that Ua is a-excessive. Also, observe from (12.321 ) that 

Ua{x) > r{x; a) + e-^^Uaia), (2.40) 

where r{x, a) is as in (12.371 ). Let u = {Ti, T2, ■■■,Ti, ^1, ^2, ii, •••} be an admissible control and let 
To = 0. Without loss of generality we will assume that r(6; a) > 0, because otherwise the corresponding 
strategy will have a lower value function J'^{x) associated to it. Since Ua is a— excessive, 

ua{x) > W [e-°^in,(XrJ] , and [e-"(^'+^)u,(X(r^+A))] " [e-"^'+in„(X7^^^ J] > 0, 

(2.41) 

for alH = 1, - 1. Then 



1=1 

N-l 

= E- [e-"^^Ua{XT^)] + E- [e-'^^^UaiXT,)] - E- 

N-l 

> J^E- [e-"^'r(Xr,,a)], 



i=l 



UaiX(^Ti+A)) 



(2.42) 



i=l 



in which the inequality follows from ( 12.401 ) and the fact that Ua is non-negative. Now, using the monotone 
convergence theorem 



(x) > E^ 


00 

E» 




J=i 


= E^ 


00 




J=i 


= E^ 


00 

E' 




J=i 


= E^ 


00 

E» 

.8=1 



E^ 



.i=l 



+ E^ 
+ E^' 



.i=l 
/•oo 

JO 



5(:e)=^.«'''(:e). 



(2.43) 



'a function / is called Q-excessive function of Xo if for any stopping time r of the natural filtration of X" and x G (c, d), 
/(a;) > E"^ [e'"^f{X°)] , see for e.g. HO) and Ql for more details. 
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The third inequaUty follows from Remark |2!T] ). The fourth inequality can be derived from (12.1 lb . The 
last equality follows from (I2.12I ). Now taking to supremum over b, we obtain ( I2.39I ). □ 

Lemma 2.6. Assume that (i2. 33i and l \2.37i hold and that the function x R{x; a) defined in ([2. 22i is 
concave and increasing on (a' ,d)for some a' £ (a, d) and that 

lim Rix-.a) = oo. (2.44) 
Thenua{x) = u"''^* (x) for a unique b* E {c,d). Hence from Lemma \2.5\ it follows that Un (x) = u°'{x) = 

Proof. Since R is concave, R'^ in ( 12.381 ) is also concave on (a', d). The assumption in ( 12.441 ) implies that 
the smallest concave majorant Wa in (12.351 ) is linear on {F{c), F{b'^)) for a unique S (c, d) and is 
tangential to R'^{-,a) at F{b'^) and coincides with R'^{-,a) on F{d)). Together with Lemma |2!2] 

this implies that there exists a unique 7* such that equations (12.301 ) and (12.31b are satisfied when W is 
replaced by Wj and b is replaced by 6'^*. Note that Wa corresponds to a strategy (a, 6'^*). That is, 
if we start with n'^'*^ and transform it via (12.191 ) we get Wj . On the other hand, using (12.351 ) with by 
substituting 7 = 7* we have that Ua{x) = ip{x)Wa {F{x)), x G (c, d). This let's us conclude that 
y^a,b'' _ ^^^2;), X G (c, d). We see that the unique b* in the claim of the proposition is b"^* . □ 

Proposition 2.7. Assume that the hypotheses of Lemma \2.6\ are satisfied. Then there exists a unique 
solution to \2.2% . Ifb* is the unique solution of \2. 29\l . then u°'{x) = u°''^* (x). 

Proof. In the proof of Lemma l2!6l we have seen that there exists a unique b* such that (12.301 ) and (12.31b 
are satisfied. Using Remark [2!2l we conclude that b* is the unique solution of ( |2.29b . □ 

Note that when the assumptions of Proposition l2.7l hold. the optimal threshold strategy is described by 
a single open interval in the state space of the controlled process. The conditions for the existence and 
uniqueness of the optimal interval are specified, essentially by the conditions on total reward function 
K{x, y) associated with one intervention from x to y (see ( 12. 14b . (|2.23b ) and drift and volatility of the 
underlying diffusion as the function F depends on them that appears in ( |2.23b depends on them. 

Third stage: Now, we let a G (c, d) vary and choose a* that maximizes p{a) and also find b* = b{a*). 
Finally, we obtain the value function given in (12.6b by v{x) = u{x) + g{x). 



2.3 Example: Optimal Exchange Rate Intervention When There is Delay 



To illustrate the procedure of solving impulse control problems with delay, we take an example from 
Mundaca and 0ksendal lITSll (also see 0ksendal |[T9]| ) that considers the following foreign exchange rate 
intervention problem: 



"'0 



(2.45) 
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where = x + i?^, in which i? is a standard Brownian motion. Here, the superscript is to indicate 
that the dynamics in consideration are of the uncontrolled state variable. In (12.451 ). c > and A > 
are constants representing the cost of making an intervention. The problem without delays are solved 
by |[T9|| through quasi-variational inequalities and by fTTl| using a direct characterization of the value 
function. In this problem, the Brownian motion represents the exchange rate of currency and the impulse 
control represents the interventions the central bank makes in order to keep the exchange rate in a given 
target window. At time Tj, such that Xt^^ = b, the central bank makes a commitment to reduce the 
exchange rate from 6 to a < 6, which is implemented A units of time later. During the time interval 
(Tj , Tj + A] the central bank does not make any other interventions. A units later if the exchange rate is 
still greater than a, then the central bank reduces the exchange rate from ^(t^+A)- to pays a cost 
of c + A(X('p^_,_^)_ — a). On the other hand, if A units of time later if the exchange rate is less than a, 
the central bank chooses increases the exchange rate to a at a cost of c + A(a — Xf^Xi+A)-)- This is a 
one-sided impulse control problem, in the sense that a control is triggered only if Xt > b and there has 
not been any previous action in the interval {t — A, t). 

The problem is to minimize the expected total discounted cost over all threshold strategies. 



voix) = inf J]^(x) 



(2.46) 



A similar version of this problem is analyzed by 0ksendal and Sulem ll22]| . in which they take the controls 
G J^T, for all i. (This introduces path dependence since the value of Xt^+a is partially determined by 

Instead of solving a minimization problem of (I2.46I ). we will solve 



v[x) 



supE"^ 



poo 

/ e— - ^e-"(^»+^)(c + Aie.l) 



and recover the value function by vd{x) = —v{x). (Here, the supremum is taken over all the threshold 
strategies.) The continuous cost rate is /(x) = — x^ and the intervention cost is K{x, y) = —c — \\x — y\ 
in our terminology. By solving the equation {A — a)v{x) = ^?;"(x) — av{x) = 0, we find that 

. Using Fubini's 



ip{x) = and ip{x) - 

theorem we can calculate ^(x) exphcitly as: 



Hence F{x) = e^^'^^o ^nd ^-^(x) 



log X 
2j2a 



9{x) 



/o 



^ 1 

'{x + Bsfds = -( — + — 



a 



We shall follow the procedure described in the last section: Let us fix a > and consider 



r (x, a) = E^ [e-"^it(XA , a)] = E 



e ( - c - A|Xa - a| + g{a) - g{X^^ 



(2.47) 



E^ 



-aA 



-c - A|x + - a| - — + ^ + 



[x + B^f 1 



a 



-aA 



c — A ( 2A exp 



(a — x)' 
4A2 



+ (a-x) -l + 2iV 



+ — 



a — X 



A 



+ 



a 
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The left boundary — oo is natural for a Brownian motion and, for any a > 0, 

r(x,a)~^ 
l-oa = limsup — = 0. 

xl-oo ^{X) 

It follows that passes through (F(— oo), /__oo) = (0,0). (See Dayanik and Karatzas fT2l Proposition 
5.12.) 

Proposition 2.8. For the function r in <\2.471l . there exists a unique solution to <\2.291l for a fixed a. 

Proof. See Appendix. □ 

Using the algorithm we described in Section |2!2] we find the optimal (a*, b*, p*). Going back to the 
original space we get 

V{x) = sup u{x) = (p{x)W*{F{x)) = (p{x){P*)F{x) = p*e''^. 

a,6eM 

on X G (— oo, b*]. To get v{x) = sup^ J'^{x), we add back g{x), 



v{x) = V{x) + g{x) = p*e"^2a _ ( ^ + i_ 



Finally, flipping the sign we obtain the optimal cost function as 



vd{x) = °^ ^ U ^ ' - - ' (2.48) 

-e-"^p*e'^*^-r(x;a*) + ^ + ^, b*<x. 



Figure 1 is obtained when the parameters are chosen to be (c, A, a. A) = (150, 50, 0.2, 1.0). We found 
the solution triplet to be (a*, 6*, p*) = (5.066, 12.1756, 0.042423). The optimal cost function without 
delay, for the same parameters, has the solution triplet (ao,6oiPo) = (5.07723,12.2611,0.0492262). 
The continuation region shifts to the left with delay (it shrinks from (— oo, 12.2611) to (— oo, 12.1756)), 
and the central bank acts more aggressively when it encounters delays (see Figure 1 - (c)). 



3 Firing Costs and Labor Demand: Optimal Band Strategies 

In this section, we will improve on the techniques of the previous section in order to study an impulse 
control corresponding to band policies when there are implementation delays. In particular, we will 
concentrate our attention on a specific example, which is of practical interest. We will find optimal 
hiring and firing decisions of a firm that faces stochastic demand and has to conform to regulatory delays 
when it is firing employees. 

Recently, General Motors Corporation (GM) has decided to lay off 25,000 of its work force to cut 
back on its production and administrative costs. However "GM's UAW (United Auto Workers) contract 
essentially forces it to pay union employees during the Ufe of the contract even if hourly workers are laid 
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vD ( X ) vD ' { X ) 




2.5 5 7.5 10 12.5 15 2 4 6 8 10 12 



(c) (d) 

Figure 1: (a) The optimal cost function vu{x). The dotted line and the solid Une fit each other continuously at 
b* = 12.1756. (b) The derivative of V]j{x), showing that the smooth-fit principle holds at b*. (c) Comparison 
of vd{x) with the cost function without delay vo{x). Note that vd majorizes vq. (d) Plot of the difference of 

vd{x) - Vo{x). 



off and their plants are closed. But those protections only run through September 2007, when the current 
four-year pact with the union ends. GM spokesman Ed Snyder said the automaker has yet to reach any 
agreement with the UAW yet on the nature or the manner of the work force reduction. "@This is a typical 
example of a firing cost and implementation delay a corporation faces when the workers are unionized. 
Another example of firing delay caused by government regulations in Europe (see e.g. [9^). 

Bentohla and Bertola 191 address the issue of costly hiring and firing and its effects on unemployment 
rate in Europe using singular stochastic control. Here, we are solving an impulse control problem since 
we are also taking fixed cost of labor adjustments into account. But our main purpose is to measure 
the effects on firing delay in decisions of firms. As we shall see, it turns out that the controlled state 
variable is not Markov, therefore we will focus our attention completely on the band policies (which 
we will define shortly) rather than trying to find the best impulse control policy. Our method of solving 
impulse control problem differs from its counterparts that use quasi-variational inequalities since we 
give a direct characterization of the value function as a linear function in the continuation region without 
having to guess the form of the solution and without having to prove that the conjectured solution satisfies 
conditions of a verification lemma. 

^Source: June 7, 2005 CNN Money, "GM to cut 25,000 jobs" by Chris Isidore, |http://money.cnn.com/2005/06/07/| 
news/fortune500/gm_closings/ 
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3.1 Problem setup 

As in ||9]R we will consider a firm with a linear production technology. In particular the quantity sold is 
Qt = ALt, A G M_|_, in which Lt is the labor at time t. The selling price at time t, Pt, of the product is 
determined from ^ 

Qt = ZtPt~, /xG(0,l) (3.1) 

in which Zt indexes the position of the direct demand curve whose dynamics follow 

dZt = Ztbdt + ZtatdWt (3.2) 

with a constant h G M+. In equation (13.11 ) the quantity 1 — ^ is the firm's monopoly power. Let us 
denote the filtration generated by the demand process Z by JT = {Tt)t>o- We will make the following 
assumption to guarantee that (13.21 ) has a unique strong solution. We assume that a is bounded and adapted 
to the filtration of the Brownian motion W. 

In our framework, if the firm produces excess products because of the excess labor, the products 
produced are still all sold but at a cheaper price. The firm pays a wage, w, to its workers, therefore the 
net rate of profit that the firm makes at time t is given by 

QtPt - wLt = zl^^iALtr - wLt. 

When the workers quit voluntarily, the firm bears no firing costs and we assume that the workers quit at 
rate 5, that is, without any intervention from the management the labor force follows the dynamics 

dL? = -SL^tdt. (3.3) 

Here, as in the previous section, the superscript indicates that there are no controls applied. The firm 
makes commitments to change its labor force at times {S'jjjgN and {TjjjgN- At time Si the firm makes 
a commitment to increase its labor force (which is immediately implemented), and at time Tj it makes 
a commitment to decrease its labor force, which is implemented A units of time later. During the time 
interval {Ti,Ti + A] the firm makes no commitments to change its labor force. Note that although at 
time Tj the firm decided to decrease its labor force, the labor force itself might move to very low levels 
following the dynamics ( 13.31 ). therefore at time Tj + A the firm may end up hiring to move keep the 
production level up. However, if the labor force level is still very high at time (Tj + A)—, then the firm 
ends up firing. Here, A represents the regulatory delays a firm faces when it is cutting off its work force. 

The labor adjustments come at a cost: At time Si the firm increases the labor by (i{> 0) G Ts^ (Here, 
for the sake of brevity we are taking the cr-algebras as a collection of mappings.) to Ls^- + Ci' then the 
associated cost is 

ciCi + C2Lsi-. 

^The set up of Bentolila and Bertola |9| was brought to our attention by Keppo and Maull. In the INFORMS Annual 
Meeting in 2004, Keppo and Maull presented their partial results on the hiring and firing decisions of firms which they obtained 
by solving quasi-variational inequalities. 
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At time Tj, the firm makes a commitment to decrease the labor at time Ti + A. If it ends up decreasing 
the labor force by r?i(> 0) G J^t,+a to Lt^+a = -^(Ti+A)- ~ Vi^ then the associated cost is quantified as 

C3r]i + CiL^T.+A)-, 

which depends on the amount of labor force to be fired and the level of the total labor force as well. The 
latter component of costs is based on the following observations: When a corporation decides who to be 
fired or which division to be restructured, administrative costs will become larger in proportion to the 
size of the total labor force since the firm's operations are closely knitted among various divisions. 

On the other hand as we discussed above if the labor force itself moves to very low levels itself during 
the A units of time, at time Tj + A the firm may end up hiring (in this case r/j < 0) to keep the production 
up at the cost of 

Cl|??i| +C2L^T,+A)- 

for some positive constants ci , C2 , C3 , C4 and A > 0. This cost becomes negligible as A becomes small 
because in that case the work force does not change much by itself. So the controls of the firm are of the 
form 

1/ = {Si, 82,- ■■ ;Ci,C2, • • • ■,Ti,T2,--- ;??i,r/2, ■■■), 

where < 5i < S'2 < • • • and < Ti < T2 < • • • are two increasing sequences of stopping times 
of the filtration J^. Tj+i — Tj > A and for any i there exists no j such that Tj < Sj < T^+a- The 
magnitudes of the impulses satisfy 0) G J^Si and r/j(e M) G J^Ti+A for all i. We call these type of 
controls admissible and we will denote the set of all admissible controls by V. To each control v & A 
we associate a profit function of the form 



J^(z,0 =1E 



00 

e-^' i zl'^{ALtY - wLt) dt-Y, e-'^' (ciO + C2L5,-) 

(3.4) 

^e-Kr,+A) (■('cgr/j +C4L(r^.+A)-) l{r,,>0} + [cir]j + C2L(^T,+A)-) 1{»?,<0} 



which incorporates the profit and cost structure we described so far. Here r > 6 is a subjective rate of 
return that the firm uses to discount its future profits. In fact if r < 6, then taking no action is optimal as 
we will point out below. Under the measure P, we have that Lq = / and Zq = z almost surely. 

The objective of the company is then to maximize its profits by choosing the best possible strategy v* 
such that 

v{z, I) = sup r{z, I) = r' {z, I), (3.5) 
if the optimal strategy u* exists. Hereafter, we will refer to v as the value function. 



It looks as if the control problem defined in (13.51 ) involves two state variables, namely the demand 
Z and the labor force L. Recall that we have no control over the demand Z but we can control the 
labor force L by making hires and fires. But the only source of randomness is the demand process. In 
the sequel we will show that the optimal control problem (13.51 ) involves only one state variable. On 
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denoting = Lt/Zt, t > and the absolute changes in labor per unit of demand by f3i = d/Zs^ and 
CKj — Vi/ZTi+A S Txi+A, we can write the the profit function J'^ as 



■ /•OD 

Jo 

(3.6) 

Let us introduce a new probability measure Pq by 



dF 



Zt^ where Zt = exp ( / GsdWs ~ ^ I CTgds 



for every < t < 00. Using the representation of the profit function J'^, we can write it as 

r{z,i)=zi'' (^), 



(3.7) 



(3.8) 



in which 



F{0 = Eo 

_'^^{b-r)(T,+A) fi^^^^, _^ C4C(T,+A)-)l{a,>0} + (ciOj + C2C(r,+A)-)l{a,<0} 



(3.9) 



where is the expectation under Pq given that ^0 = C- Here, with slight abuse of notation, on the 
right-hand-side of ( 13.81 ). we denoted 

= {Si,S2, ■■■ ; /3i,/32, • • • 'iTi,T2,--- ; 01,02, ■■■), 

is a control that is applied to the process ^. The controls here are such that A ( > 0) £ .7^5". and ( € M) G 
Txi+A- Again as before {/SnjneN and {r„}„gN are two increasing sequence of stopping times. We also 
assume that Tj+i — Tj > A > and that for any i there exists no j such that Tj < Sj < Ti^^- With 
another slight abuse of notation we will denote the admissible set of controls we described here also by 
V. As a result of the developments in the last part of this section we see that the process Lt/Zt is the 
sufficient statistic of the problem in (13.51 ). In fact we can write the value function as 



(3.10) 



v{z,l) = zY (y) , where Y{i) = supr(e). 



Under the measure Pq the dynamics of the process, when there are no impulses applied follows 



= e exp (b + 6)t - asdB, - ^ a^ds^ 



(3.11) 



where i? is a Wiener process under measure Pq. Here, as before, the superscript indicates that there are 
no controls/impulses applied. 
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3.2 Solution 



Although the controlled process ^ is not a Markov process, because depending on whether the process 
reaches a point during the interval (Tj, Tj + A) or not, that point has different roles. That is, how the 
process reaches to a particular point (path information) affects how the process will continue from this 
point. However, the process regenerates at times {Tj + A}jgi!^ and the value of the process at time 
T G (Tj, Tj+A)> Xt, depends on the information up to Tj, JF^-., only through the value of the process at 
time Ti, Xt^- Therefore, as we did in Section ITTl assuming there is no history prior to time 0, i.e. J^q is 
a trivial sigma-algebra, we can develop 



where C2(x,y) = -ci(y - x)l{j,>^} - C2X, and, 

Ci{x,y) = -{c3{x-y) + C4x)l{^yyy + C2{x,y)l{y~^^} 



5(e) =lEo 



,{b-r)t 



On denoting u{^) = — g{i), we can write 

U{0=^1 flm<5i}e('-'^)(^^+^) (Cl(C(T,+A)-,eT,+A) +n(eT,+A)) 



in which 



Ci{x,y) = Ci{x,y) - g{x) + g{y) and C2{x,y) = C2{x,y) - g{x) + g{y). 



(3.12) 

(3.13) 
(3.14) 



(3.15) 



(3.16) 



In the rest of this section, we will analyze the following double sided threshold strategy {band policy) 
of the following form: 1) Whenever the marginal revenue product of labor hits level d, the firm makes 
a commitment to bring the marginal revenue product of labor to c < d. This may be achieved by firing 
employees if marginal revenue product of labor is still greater than c after the delay. However, it is 
possible that after the delay the marginal revenue product of labor will be less than c. In this case, the 
firm makes hires. 2) Whenever the marginal revenue product of labor hits level p the firm increases it to 
q > p (by hiring new employees). We will characterize the value function corresponding to an arbitrary 
band policy. 

For a band policy we described above Si = Tp and Ti = r^, and 

Cti+a = ^in+Ay -ai = c and = Csi- + Pi = q. 
Here, for any x € M+, = mi{t > : = x}. Let us introduce 

no(C) =IE^[e(^-'^)^n|,^<,^p(d)] +E«[e(''-^')-n|,^>,^p(p)], (3.17) 
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in which 



u{d)=Kt\e^'-'^^{Ci{e^^,c)+u{c))\ and u{p) = C2{p, q) + u{q). 



(3.18) 



From (I3.15I )- (I3.18I ) it can be seen that 



C2i£,,Q) +uoiq), C<P; 
r(C,c)+e(*-'^)^txo(c), C > d. 



in which 



(3.19) 



(3.20) 



Let us denote the fundamental solutions of {A + {b — r))f = 0, hy ip (increasing) and ip (decreasing), 
and introduce F = ip/ip. Using (I2.18I ). on the interval {p, d) we can write u as 

u{0 uid) {FiO - F{p)) , u{p) {F{d) - FiO) 



+ 



^{0 m (Fid) - Fip)) ^(p) (Fid) - Fip)) 



(3.21) 



Then, W 



A « ^ 77-1 



o F , satisfies 



Using the linear characterization (in the continuation region) of the band policies in ( 13.221 ). the following 
algorithm first determines the function u for an arbitrary band policy and goes onto finding the best band 
policy. 



First, let us define 



Ri{x;c) 



F-^{x) and R2{x;q) 



C2{;q) 



F-\x). 



(3.23) 



Algorithm: 



1. For a given band policy which is characterized by the quadruplet {p, q, c, d) such that p < q < c < 
d, we can find the value function u in ( 13.191 ) using the linear characterization in (13.221 ). On [F(p), 
F(d)] we will find W{y) = py + t (in which the slope p and the intercept r are to be determined) 
from 



,(fe-r)A 



ipF{c) + r 
{pF{q) + T 



+ Ri{F{d);c)= pF{d)+T, 



R2{F{p);q)= pF{p)+T. 



p and T are determined as 



R2(F(p);q) (^(b-r)A V>{c) 
l-ip(q)/ip{p) 



^-1 +R,{F{dy,c) 



p(^Fiq)-F{p))+R2{F{p-q)) 



(3.24) 



(3.25) 



19 



Now u can be written as 



Mq) + r2{C,q), x<p, 
e(^-'-)^uo(c) +ri(e,c), x > d. 



(3.26) 



From this last expression, we observe that {A+ {b — r))u{^) = for ^ G (p, d). 

2. Note that p and r are functions of {p, d) parametrized by {q, c). We will find an optimal pair (p, d) 
given (g, c) by equating the gradient of the function {p, r) with respect to {p, d) to be zero. Now, 
differentiating the first equation in (13.241 ) with respect to d, and the second with respect to p, and 
evaluating them at Td = Pd = Tp = pp = we obtain 



(pFiq) + T)^^'{p) - pF'ip) + ^R,{y- q) 
ip{pY dy 



y=F{p) 

- e(''-'-)^(pF(c) + r)^^\d) - pF\d) + ^R,{y; c) 



F'{p) = 



'^{df 



dy 



(3.27) 



y=F{d) 



F'{d) = 0, 



in which p and r are given by (13.251) . To find the optimal [p, d) (given (c, q)) we solve the non- 
linear and implicit system of equations in (13.271) . 



Remark 3.1. On [F(0), F{p)\ the function W is given by 

iyw = ((,f(<,) + r)^l|(|L_ 

and its left derivative at F(p), W'{F{p)—), is given by 

f{q) (f'{p) 



+ R2{x;q), 



(3.28) 



W'{F(p)-)=-{pF{q)+T) 



+ 



d 



ifipf F'{p) dy 



(3.29) 



y=F{p) 



Therefore, the equation in H3.27\) in fact implies that the left and the right derivative ofW at F{p) 
are equal (smooth fit). (Recall that W{x) = px -\- ry on [F (p) , F (d)].) Similarly, the second 
equation in ([3.27i implies that the left and the right derivative ofW at F{d) are equal. This can 
be also expressed as: "R2 shifted by an appropriate amount is tangential to the line I (y) = py+r" 
at F{p). 



3. Next, we vary q and c to find the best band policy. Such a search can easily carried out in Mathe- 
matica. 



To obtain an explicit expression for g in (13.141 ) and r in (13.201 ) we make the following assumption. We 
will assume that at = a > Q (a. constant) in (13.21) . Now, we can obtain g in (13.141) (see Appendix) 
explicitly as 

= — 2 r^^^ - = fci?'^ + k2i. (3.30) 

r -h + {b + 5)p + ^a^p- ^a^p^ r + o 



20 



Note that if r < 6, then g{£^) = oo, which imphes that taking no action is optimal. The assumption 
in Proposition 13. 1 1 that max(ci — 02,03 + 04) < \k2\ is for technical reasons, however it is not very 
restrictive. k2 denotes the present value of the total wage that a firm pays per unit of marginal revenue 
product of labor and it should be greater than costs associated with one time hiring or firing of one unit 
of marginal revenue product of labor. Using (13.301) we can also calculate r in (13.201) explicitly as (see 
Appendix) 



c) = e^'-'^)^ - (C3 + C4)e-('+^)^eA^(di) + (ci - C2)e-('+'^)^CA^(-di) 
+ C3cN{d2) - cicN{-d2) - h exp (e) - fcze-^^+^^^C + hc^ + he 

in which 



(3.31) 



Here the function x — > N{x), x G M, denotes the cumulative distribution function of an A^(0, 1) 
(standard Gaussian) random variable. The infinitesimal generator A of the process ^ is Au{x) = 
(cj^/2)x^'u"(x) — (6 + 6)xu'{x), acting on smooth test functions u{-). Therefore the fundamental solu- 
tions of the equation {A + {b — r))u = are 

ip{x)=x'^\ p = x^\ (3.33) 

in which /3i > 1 and /32 < are the roots of the following quadratic equation (in terms of fi) 

^ct2/52 - + (6 + 5)^ /3 + 6 - r = 0. (3.34) 

The next proposition justifies the second stage of our algorithm. 

Proposition 3.1. For a given {q,c) £ M?, such that {ciq — {kiq^ + k2q)) < there exits a unique 
solution iv* ,d*) to the system of equations i[3.27\) if we further assume that maxfci — c?, + Ca) < |/c2|- 
Moreover, u^* ''''^''^* (x) = supQ^p^^u^''''^'^ (x) , x >0. 



Proof. The proof is similar to that of Proposition 12.81 Also, see the remark below. □ 

Remark 3.2. The proof of Proposition \3. l\ onl\ relies on the following properties of the functions Ri and 
R2 defined in ^3.23^ : 1 ) There exists a point j £ (0, cxd) such that y — > Ri{y;c) is concave and increasing 
on (j, cxd); 2) limj^^oo Riiv, c) = 00; 3) The function y R2{y, q) is increasing and concave on (0, t) 
for some t < F{q) and decreasing on (t, 00); 4) Both y — > i?i (y; c) and y — > R2{y, q) are differentiable. 
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Our results in this section can be generalized to the two-sided control of any one-dimensional diffusion 
and penalty functions satisfying the conditions in Remark 13.21 are satisfied. It is worth pointing out 
that Weeransinghe [25 1 has studied the two-sided bounded variation control within the framework of 
singular stochastic control of linear diffusions for a large class of cost functions by using of the functional 
relationship between the value function of optimal stopping and that of singular stochastic control (see 
e.g. Karatzas and Shreve HU). 

3.3 Numerical Example 

In this section, we will give a numerical example for the labor problem with and without delay. We select 
the parameters as 6 = 0.03, r = 0.06, fi = 0.75, a = 0.35, 6 = 0.1, A = 5,w = 2,A = 0.5, ci = 0.05, 
02 = 0.1, C3 = 2 and C4 = 1. The results we obtain are summarized in the following table: 





P 


r 


P 


q 


c 


d 


A = 


0.0002003 


38.1633 


1.0664 


2.125 


7.240 


35.728 


A = 0.5 


0.0001725 


38.1597 


1.0661 


2.100 


7.120 


36.640 



Both the slope p and the intercept r are greater in the no-delay case and therefore, the value function 
corresponding to no-delay problem {x) will dominate that to delay problem v^{x). On the right 
boundary, we have (7.240,35.728) C (7.120,36.640) and on the left boundary (c, d) pair has shifted 
to the left with delay. As a result, the continuation region {p, d) has expanded with delay: = 
(1.0664,35.728) C (1.0661,36.640) = C^. An explanations for this phenomenon can be made through 
the relative size of costs of firing and hiring, the size of delay parameter, the shape of g function, etc. 
In our example, the firing cost is relatively larger than hiring cost, the penalty of firing becomes smaller 
with delay (than without delay) which encourages the controller not make hasty firing decisions, facing 
relatively large firing costs. Or since there is a chance that the process moves to the left during the delay 
period due to voluntary quits, this effect may help to reduce firing costs even though the decision making 
is postponed. 



4 Conclusion 



In this paper we give a new characterization of the value function of one-sided and two-sided impulse 
control problems with implementation delays. We also provided easily implemented algorithms to find 
out the optimal control and the value function. Our methodology bypasses the need to guess the form of 
solution of quasi-variational inequalities and prove that this solution satisfies a verification lemma. Since 
our method directly finds the value function, we believe that this method can solve a larger set of problems 
than just with quasi-variational inequalities. Indeed, we applied our results to solving some specific 
examples. As an important application of a two-sided impulse control problem with decision delays 
we found out the optimal hiring and firing decisions of a firm facing regulatory delays and stochastic 
demand. 
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Figure 2: (a) The graph of g{x). (b) The graph of r{x, c*) for A > (c) The graph of hne p*y + t* we obtain 
via our algorithm and Ri{y, c*) after it is shifted vertically by e^''^^'>'^{pF{c) + r) ^(^p-f(^yyj ■ (d) The graph of 
the line p*y + r* and R2{y, c*) after it is shifted (see ( 13.281) for the amount of shift), (e) The two value functions, 
v^{x) (A = 0) above and v°{x) (A > 0) below, (f) Plot of difference, {x) - v^{x). (f) Plot of difference, 
(x) — v^{x). (g) (h)The derivatives match atx~p and a; ~ d (A > 0). 
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Here we considered a problem in which the decision maker needs to decide whether to take action and, 
after some delay, needs to decide the magnitude of her action. In the future, we will consider problems 
in which the decision maker takes action and waits that action to be implemented. We will also consider 
a general characterization of the value function and the optimal controls when the decision delay is not a 
constant but it depends on the magnitude of the action taken as in ll24l or it depends on the value of the 
state variable that is controlled as in [.3,1. 
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5 Appendix 



5.1 Derivations of (IXIib and (I33I1) 

Using (13.111 ) we can write (13.141 ) as 



r 



^/^e^^-^)* exp(-(6 + 6)fit - afiBt - -a^fit)dt 



A''^'' I exp 
/o 



f°° 1 
/ (,exp{-{b + 6)t - aBt - -a'^t)dt 
Jo 2 

/•oo 

dt-wi I exp{-{b + 5)t)dt, 
Jo 



t ( 6 - r - (6 + - + V 



(5.1) 



from which we obtain (13.301 ) under the assumption that r > b. Here the second inequality follows from 
the Fubini's theorem and using the Laplace transform of Bf. 

In what follows we will present the derivation of (13.311 ). We can write (13.201 ) as 



-C3(Ca - c) - C4Ca) 1{Ca>c} + (-ci(c - Ca) - C2Ca)1{^a<c} 



h^A - ^sCa + hc^" + k2C 



(5.2) 



Using (13.111 ) and the assumption that at = a € M+, we compute 



^ ^ [1 



{Ca>c}J 



N{d2), B^El[l{^^^,y]=l-A = Ni-d2 



C{e) ^ E? I = e^xp ( - ( 6 + 5 + - 0) ) ) , 



(5.3) 



where 9 = \ or 9 = n. Here the third equality follows from the Laplace transform of Bt We will also 
need to compute 
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We will denote 



K = exp ^— -cr^A + (T\/A??^ , 



in which i] = B^/\' A, is an A^(0, 1) random variable. Then = '?exp(— (6 + 6)A)k and A = 
A>c}'^J- Introducing a new probabihty measure Q by the radon-nikodym derivative 
dQ^/dPQ = K, we get 

D = e-(^+^)^eQ^(CA > c). 
Under the measure Q^, n = —rj — a\fK is (0, 1) and we can write in terms of n as 

^A = eexp (^-(6 + 5- ^cj2)A + aVAn^ . (5.5) 

Using (15.5b . we can compute 



D = ^e-(^+^)^iV(di), (5.6) 
in which di is given by (13.321 ). We can then immediately obtain, 

E 4 E« [eAl{5A<c}] = ^e-(^+'^)^(l - Q«(eA > c)) = ee-(^+'^)^iV(-di). (5.7) 

Using (E2]l, (ES), (ISSi and we obtain (ITIOl) . 

5.2 A Technical Lemma 
Lemma 5,1, Define 

G(x,7) = supE^[e-"^(/i(X°) +76""^)], x G M, 7 G M, 
/or i'ome Borelfimction h. Then for 71 > 72 we have that 

G(rE,7i) - G(x,72) < 71 - 72- 

Proof. See the proof of Lemma 3.3 in ifTTl . □ 



5.3 Proof of Proposition 

The proof follows from the analysis of the function r. The following remark will be helpful in the 
analysis that follows. 

Remark 5,1, Let us denote H{y) = (h/ip) o (F^^(y)), y > 0. If h{-) is twice-differentiable at x G X 
and y = F{x), then H' (y) = m(x) and H" (y) = m {x)/F' (x) with 

mix) = -^(-) (x), and h" {y)[{A - a)h{x)] > 0, y = F{x), (5.8) 
F (x) \^pj 

with strict inequality if H" (y) / 0. 
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5.3.1 The Analysis of the Function r in ( I2.47D 



Let us check the sign of (x) = ^ '^^/'^ {x) which is the same as the derivative of R as can be 
observed from the first equation in ( I5.8I) . The sign of (j^^ (x) is the same as that of 

^2 

4A2' 

1 / {a-x)^\ 1 



- + A - 2aAA exp ( - . ^' 1 - ca 



a 



+ A(a-x) -4exp +4^rV^ +^/2^ 2iV 



+ - + Af2ivf^)-1). (5.9) 



A ^ V 4A2 ; ' A^v A ; ' V V a 

Using the fact 27V < 1 for x > o and exp (-■^^^) + (^) < for x > a sufficiently 

large,in this equation (for sufficiently large x) we identify the absolute value of the negative terms as 
^AAexp (-^^^) < ^AA , ca and |A (2iV (^) - l) | < A. Since these negative terms 
are bounded, if we take sufficiently large value, say a', the sign of (15.91) is positive for x G (a',oo). 
Moreover, we can directly calculate liniy^+oo ^^(y; a) = to check the behavior of i?(y; a) for a 
large y. We also know that a) = (r(-, a)/^p{-)) o F^^{y) is negative at y = -F(a). On the other 
hand, l/(p{F^^{y)) = ^ is increasing and concave function. It follows that i?(y; c) + ^(^p-i (^y-^-^ is an 
increasing function on y G (F(a'), oo). 

To investigate the concavity of R{y; a), we set 

Q{x,a) = -x—{e a M — — _ 3^ _ A(a - x)(/) ' 



2aV V 4A2y "^V^y '"^V^ 

+ ax^ — ar(x, a) 

so that — a)r(x, o) = g(x, o) for every x > 0. We have lim2,._+oo q{x) = —00 if a < 4. By the 
second equation in ( 15.81) . the function R{y; a) becomes concave eventually. Since R{-; a) is increasing 
and concave on {a", 00) for some a" > a' and limy^oo R{y', a) = 00 we can find a unique linear 
majorant to R'^{-,a) in Lemma 1231 (the linear majorant majorizes R'^{-, a) in the continuation region and 
is equal to R'^{-,a) in the stopping region). The rest of the proof from Proposition 
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